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Abstract 

Zuckermann [10] considers the problem of optimal control of a finite dam 
using P*Q policies, assuming that the input process is Wiener with drift term 
H > 0. Lam and Lou [7 ] treat the case where the input is a Wiener process with 
a reflecting boundary at its infimum, with drift term /i > 0, using the long-run 
average and total discounted cost criteria. Attia [3] obtains results similar to 
those of Lam and Lou, through simpler and more direct methods. Bae et al. 
[5] treat the long-run average cost case when the input process is a compound 
Poisson process with a negative drift. In this paper we unify and extend the 
results of these authors. 
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1. Introduction and summary 



Lam and Lou [7] consider the control of a finite dam, with capacity > 0, 
where the water input is a Wiener process, using policies. In these policies 
the water release rate is assumed to be zero until the water reaches level A > 
0. As soon as this happens the water is released at rate M > until the water 
content reaches level r > 0, A > r. They use the total discounted as well as the 
long-run average cost criteria. Attia [3] obtains the same results of Lam and 
Lou, using simpler methods. Lee and Ahn [8] consider the long-run average cost 
case, for the PJ^q policy, when the water input is a compound Poisson process. 
Abdel-Hameed [1] treats the case where the water input is a compound Poisson 
process with a positive drift. He obtains the total discounted as well as the 
long-run average costs. Bae et al. [4 ] consider the P^q policy in assessing 
the workload of an M/G/1 queuing system. Bae et al. [5] consider the log- 
run average cost for P*^ policy in a finite dam, when the input process is a 
compound Poisson process, with a negative drift term. At any time, the release 
rate can be increased from to M with a starting cost KiM, or decreased 
from M to zero with a closing cost K2M . Moreover, for each unit of output, a 
reward R is received. Furthermore, there is a penalty cost which is a bounded 
measurable function on the state space of the content process. In this paper 
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wc treat the more general cases where the input process is assumed to be a 
spectrally positive Levy or a spectrally positive Levy process reflected at its 
infimum. 

For any process Y = {Yt,t > 0} with state space E, any Borel set A 
C E and any functional /, Ey{f) denotes the expectation of / conditional on 
Yq = y, Py{A) denotes the corresponding probability measure and 1a{ ) is the 
indicator function of the set A. In the sequel we will write indifferently Pq or 
P cmdE qOyE. Throughout, we let R = (-00,00), R+ = [0,oo), = {1,2,...} 
and A''+ = {0, 1, ...}. For x,y G R, we define x\/y = a;maxy and xAy = xminy. 

For every t>0,we define Y^ = inf (Fs,AO), Yt = sup (F^, VO). 

0<s<t 0<s<t 

We will use the term "increasing" to mean "non-decreasing" throughout this 
paper. 

In Section 2, we discuss the cost functionals. In Section 3 we define the 
input processes and discuss their properties. In Section 4 we obtain formulas 
needed for computing the cost functionals using the total discounted as well 
as the long-run average cost cases. In section 5 we discuss the special cases 
where the input process is a Gaussian process, a Gaussian process reflected at 
its infimum and a spectrally positive Levy process of bounded variation. 

2. The cost functionals 

For each t G R+, let Zt be the dam content at time t , Z = {Zt,t G R+}- We 
define the following sequence of stopping times : 

To = mi{t>0:Zt>X}, Tq = ini{t > Tq : Zt = t}, 

Tn = inf {i > T„_i : Zt > A}, T„ = inf {i > T„ : = r}, n = 1, 2, ... (2.1) 

It follows that the process Z is a delayed regenerative process with regenera- 
te 

tion points {T„,n = 0, 1, ...}. The regeneration cycle is defined to be the time 
between successive regeneration points. During a given cycle, the release rate 
is either or M. When the release rate is zero, the process Z is either a 
spectrally positive Levy process (denoted by /) or a spectrally positive Levy 
process refiected at its infimum (denoted by Y), and remains so till the water 
reaches level A; from then until it reaches level r the content process behaves 

(M) 

like the process / = I — M reflected at F , we denote this process by /. It 
follows that, for each t>0, 

* (M) (M) 

It= It - sup (( / ( - 1/) V 0). (2.2) 

0<s<t 

When the release rate is 0, the dam is maintained at a net maintenance cost 
rate g, where 5 is a bounded measurable function on (I, A), where I is the lower 
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bound of the state space of the process /. Furthermore, maintenance of the 
dam when the release rate is M is done at a cost rate g* , where g* is a bounded 
measurable function on (r, V] . 

A 

For a; < A and a € R+, the discounted cost during the interval [0,To), 
denoted by Ca{x, A, 0), is given as follows 

A 

C^{x, X,0) = E, [ " e-"*g{Zt)dt. (2.3) 
Jo 

* * A 

Define, T~t = Tq — Tq, when the release rate is M, starting at a; > r, 

* 

the expected discounted cost in the interval [0, T~t), denoted by Ca{x, r, M) is 
given as follows 



Ca,ix,T,M) = E^ [ e-'''g*iZt)dt. (2.4) 
Jo 

We now discuss the computations of the cost functional using the total 
discounted cost as well as the long-run average cost criteria. Let {x) be the 

expected cost during the first cycle, [0,To), when Zq = x. From the definition 
of the policy, it follows that for A < a; < F 



at , 



C„(a;) = M{Ki - RE^ [ " e~"'dt} + Q (M, a;, r), A < a; <V (2.5) 
Jo 

and for a; < A 



C^{x) = M{K2 + K, E,[e-^^-] - ^{Eje-"^^] - i^Je""^"] } 
+a(a;,A,0) + £;je-"^«C„((ZA AV),t,M)]. (2.6) 

To 

Let Ca{X,T) and C(A,t) denote the total discounted cost and the long- 
run average cost, respectively. By modifying the result in Abdel-Hameed [1], it 
follows that 

C„(A,r)=C.(.0 + ^l^^^^^=^^^^^ (2.7) 
l-i;^[exp(-aTo)] 

and 

M{K + REo[T+_^]) + C„{x, A, 0) + E^ [C„{{Z. A V), r, M)] 

C{X,r) = ^ RM. (2.8) 

Er[To ] 
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where K = Ki+ K2 and Cq;(t) is the total discounted cost during the interval 

* 

[0,To), given that Zq = t. 

3. The input processes and their characteristics 

In this paper we consider the cases where the input process is a spectrally 
positive Levy process, and a spectrally positive Levy process reflected at its 
infimum. In the remainder of this section we describe these processes and discuss 
some of their characteristics. The reader is referred to [6] for a more detailed 
discussion of the definitions and results mentioned in this section. 

Definition. A Levy process L = {Lt,t > 0} with state space R is said to 
be spectrally positive Levy process, it has no negative jumps. 
It follows that, for each 6 G R+, x G R, 

where 

n2 2 rco 

m = -ae+^-J^ (1 - e-^- - exl{,^iy)vidx). (3.1) 

The term a G R, € R+ are the drift and variance of the Brownian motion, 
respectively, and w is a positive measure on (0, 00) satisfying J^{x^ A l)v{dx) < 
00. 

The function (j) is known as the Levy exponent, and it is strictly convex and 
tends to infinity as 6 tends to infinity. For a G R+, we define 

77(a) = sup{6l : (/){e) = a} (3.2), 

the largest root of the equation (f){d) = a. It is seen that this equation has at 
most two roots, one of which is the zero root. Note that, E{Li ) = xv{dx) + 
jjL. Furthermore, lim = 00 if and only if E{Li ) > 0, and lim = —00 if 

and only if E{Li ) < 0. Also, if E{Li ) = 0, then lim Lt does not exist. 

Furthermore, rj{0) > 0, if and only if E{Li ) > 0. 

An important case is when the process L is of bounded variations, i.e., (7^=0 

and J^{xA l)v{dx) < 00. Let 




In this case we can write 

POO 

(j){e) = Ce- (1 - e-^^)w(da;), (3.3) 
Jo 

where necessarily ( is strictly positive. 
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Definition. A Levy process is said to be spectrally negative if it has no 
positive jumps. 

A 

For any spectrally positive Levy input process L, we let L = — L throughout 

A 

this paper. It is clear that L is spectrally positive if and only if the process L is 
spectrally negative. 

We now introduce tools, which will be central in the rest of this paper. 

Definition. For any spectrally positive Levy process with Levy exponent 
(p and for a > 0, the a— scale function : R -» W°'{x) = for every 
X < 0, and on [0, oo) it is defined as the unique right continuous increasing 
function such that 



r 1 

/ e-^^W^"\x) = — ,/3>v{a) (3.4) 

Jo np) - a 

We will denote W'^ by W throughout. For a > 0, we have (see (8.24) of[ 6]) 

oo 

W^"\x) =^a''W<''+^'>{x), (3.5) 

fe=0 

where W*^''^ is the kth convolution of W with itself. 

It follows that W^"\0+) = if and only if the process L is of unbounded 
variation. Furthermore, W" right and left differentiable on (0, oo). By {x), 
we will denote the right derivative of W^"^ in x. 

The adjoint a— scale function associated with W^"^ (denoted by Z^"^) is 
defined as follows: 

Definition. For a > 0, the adjoint a— scale Z^°'^ : [i-,oo) is defined 

as 



Z'^°'\x) = l + a [ W°'{x)dx. (3.6) 
Jo 
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It follows that as a; oo, for a > 0, , W^°'\x) ~ ^^^^ and ~ 

4. Basic results 

A A 

To derive Ca{x, A, 0) , Ex[e~°''^"], C(A, r), Er[To] wo define the input process 

A 

Z killed at Tq, denoted by X such that for every t>0 

Xt = {Zt,t<To}. (4.1) 

It is known that this killed process is a strong Markov process. 

For the input process X, any Borel set A contained in the state space of 
X,tG R+, the probability transition function of this process is given as follows 

Pt{x,A)=P^{ZteA,t<To} 
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and for each a e -R+ its a— potential is defined as follows 



U"ix,A)= Ptix,A)e-''Ut = E, e-'^'l^z.eAjdt. {4.2) 
Jo Jo 



We note that for a; < A 



Cc,{x,X,0) = U"g{x). (4.3) 



The following lemma will be used extensively throughout this paper. 

Lemma. Let S = {St,t > 0} be a strong Markov process. Define, Q = 
{a{Su , u < f)}j>Q, K to be any stopping time with respect to G and U"''^{x, A) as 
the a— potential of the process S killed at k. Denote the state space of this 
process by E. Then, for x G E 

E^[e-'^'^] = l-aU^f{x). (4.4) 

Proof. From the definition of U"'^ and for any bounded measurable func- 
tion / whose domain is E, we have 

U'^'^fix) = E^r e-"'f{St)dt]= [ f{y)U''''{x,dy)- 
Jo Je 



Taking / to be identically equal to one, we have 

iii^a = t/--(x). 

The required result is immediate from the last equation above. ■ 

A 

First we consider the case where, during the period [0, Tq), the process 
Z is a spectrally positive Levy process, denoted by /. In this case we note 

A 

that To = inf{i > : 7* > A} and we will denote it by T+ . Throughout 
the rest of this paper, for any a G R, we define T~ = mf{t > : It < a}, 
T+ = inf{t >0:lt > a} and T" = inf{t > : < a}. 

(1) 

Proposition. For a > 0, a < A the apotential (t/") of the process I killed 
at T = r^A T~ is absolutely continuous with respect to the Lebesgue measure 
on [a, A] and a version of its density is given by 



u'^{x,y) = W^'^HX - x) _ - W^"\y - x), x,yG[a,X]. (4.5) 

Proof. For A c [a, A] 
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U{x,A) = [ e-"*I[i^eA}dt 
Jo 

■f 

Jo 
I ° 



I e-"*I A dt 

{It&-A} 

/■To"AT+_„ 

Ex-^ / e-"*I A dt 

' {ite\-A} 

W'^"'>{\~ a-y) 

[VV ^ ' [A — X' 



where the last equation follows from Theorem 8.7 of [7], this establishes our 
assertion. ■ 

Corollary. For a > the a-potential {U") of the process X is absolutely 
continuous with respect to the Lebesgue measure on (— oo, A] and a version of 
its density is given by 



u"{x,y) = W^"\X-x)e-''^"^'^^-y^ -W'^"\y-x), x,ye{-oo,X]. (4.6) 

Proof: The proof follows from (4.5) by letting a — )■ — oo and since, for a > 0, 
W''°'Ux) ~ as a; oo. ■ 

With the help of the last corollary above we are now in a position to find 
E^e-'^^^] and E^[T+]. 

Theorem, (i) For a > and a; < A wc have 

EJe-''^^] = Z^'^UX -x)- -^W^'^UX - x). (4.7) 

(ii) For a; < A we have 

EAT^] = ^^^'^""^ -W{X-x), r?(0)>0 (4.8) 
= oo ,??(0) = 0, 

where for every a; > 0, 

W(x) = [ W{y)dy. (4.9) 



Proof. We only prove (i), the proof of (ii) is easily obtained from (i) and 
hence is omitted. Let be the a- potential of the process X, then 
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J —oo 

= l + aj W°'{y-x)dy-aW°'{X-x) [ e-^^-^^^'^^rf?/ 

J X J — ( 



CO 

a 



where the first equation follows from (4.4), the second equation follows from 
(4.6), the third equation follows since W^"'\x) = 0, x < 0. and the last equation 
follows from the definition of Z'^°'\ ■ 

For any Borel set B C R+ x R, we let M{B) be the Poisson random measure 
counting the number of jumps of the process J in i? with Levy measure v, where 
if B = [0,t) X A, A (Z R, then E[M{B)] = tv{A). We need the following to 
compute the last term in (2.6). 

(1) 

Proposition. For a > let u°'{x, y) be as given in (4.5) and x <X< z, then 



E^le-""^^ €dz, T+ <T-]= v{dz - y)u°'{x,y)dy (4.10) 

J a 

Proof. For a; < A, a > 0, C C [A, oo) and D c (a. A) 
£,[e-«^^,Xy+ e C,X^+_&D,T+ <T-] 



= E4 e-«*I _ I{y^c-x,-}M{dt,dy) 

i[0,cx))x(0,oo), {Xt-<X,Xt-,>a,Xt-eD} 

= E^f e-"*I _ I{x,eD}HC - Xt)dt] 

i[0,oo) {Xt-<X,Xt-,>a}, ^ * < ^ 

= E,[[ e-^^^TyMx^enyi^iC - X,)dt)] 

J[0,co)), 

= E.il e-''n{,^T}AC - Xt)I{x,eD}dt)] 

J[0,oo) 

= E^[ e-"*I{t<T},z^(^-y)I{x,edj/}rfi] 

J[0,oo)xD 
f (1) 

= / T^iC -y)u"{x,y)dy, 

JD 

where the second equation follows from the compensation formula (Theorem 
4.4. of [7]). Our assertion is proved by taking = [a. A]. ■ 
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The following corollary gives the formula needed to compute the last term 
of (2.6), when the input process is a spectrally positive Levy process. 
Corollary. Let u^be as defined in (4.6). For a > and for a; < A < ^, 

+ 

[e""'^^ , &dz]= I v{dz - y)u°'{x, y)dy. (4.11) 

J —CO 

Proof. The proof follows immediately from (4.6) and (4.10). by letting 
a — >■ —CO. ■ 

A 

We now turn our attention to the case where, during the period [0, Tq), the 
process Z is a spectrally positive Levy process refiected at its infimum, denoted 

A 

by Y. We will denote Tq by , in this case. 

The following proposition gives the a— potential of the process X defined 
in (4.1). 

A 

Proposition. Assume that during the period [0, To), the process Z is a spec- 
trally positive Levy process reflected at its infimum. Denote the a- potential 

(2) 

of the process X by U". Then for any x,y & [0, A), 



U"(x,dy) = - ^ , ' ^-^-W^''\y-x)dy, (4.12) 

where for x,y e [0, A), W(«)(dy) = H^(«)(0)<5o(dt/) + W^^' {y)dy, and <5o is the 
delta measure in zero. 

Proof. Note that for each t > 0, 

Yt = It -It (4.13) 

A A 
= h-h, 

where the process / = {hit > 0} is a spectrally positive Levy process. The 

A 

result follows from part (ii) of Theorem 8.11 of [7], since the process 7 is a is 
a spectrally negative Levy process. ■ 

The following proposition gives Ex[e^°''^>-] and S^fr^^]. 
Proposition, (i) For a > and x < A we have 



^^[e-""A] = Z(")(A -x)- VF(")(A - x) "^^^),;^ ; . (4.14) 



^qW(")(A) 

(ii) For a; < A we have 



E.[r,] = W{\-x)^^^^-W{\-x). (4.15) 
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Proof. The proof of part (i) follows from (4.4) and (4.12), in a manner 
similar to the proof of (4.7). The proof of part (ii) follows from part (i) be 
direct differentiation. We omit both proofs. ■ 

To find a formula analogous to (4.11), for the spectrally positive Levy process 
reflected at its inflmum, we flrst need few deflnitions. For 2; > A we let 



..A 

lc{dz) = W'^"\\-x) / W'^°'{dy)v{dz-y) (4.16) 

Jo 

-Tvj"' (A) / dyW'-°'\y - x)v{dz - y)]. 



La{z) = / Udu). (4.17) 

J (2,00) 

V^{X) = W^|")'(A)Z(")(A-a;)-aW^(")(A-a;)W^(")(A). (4.18) 

The following proposition gives the required formula. 
Proposition, (i) For a > and for a; < A < 2, 

EJe— sn^ed.] = ^;^g^,.>A. (4.19) 



i^Je— sF.^ = A] = ^"r/w^:,"; • (4-20) 



(ii) For a > 

V a{X)-L^iX ) 

Proof, (i) From (4.13), for a; > 0, Yb = x if and only if Iq = x if and only 

A 

if lo = —X . Furthermore, = 1^+ almost surely on {T^ < Tq }. Therefore 

[e-""^ , e dz}] = [e-"-A , Y^^ e dz, T+ < T^] + [e'^-A , Y^^ e dz, T+ > 
= E^ [e-^^^ , e dz, T+ < To"] + E, [g-^^o" ,T+ > Tq"] 

xE^[e-"^^,Yr^ €dz] 
= E^ [e-^^^ , 7^+ e dz, T+ <T^]+E_, [e-^^o , TI^, > T+] 

xE,[e-"^x,Yr^ €dz] 
= E^ [e-«^A+ , 7^+ e dz, T+ < To"] + Ex_^ [e'^^^ , Tq > T+] 

xE^[e-"^^,YrGdz], (4.21) 



where the second equation follows using the strong Markov property, the third 

A 

and fourth equations follow from since 7 = —7 and from the definitions of 
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Letting a ^ in (4.5) and (4.10), we find that the first term in the last 
equation above is equal to J v{dz — y)[W'^"\X — x) — W'^°'\y — x)]dy. 



The second term is equal to (see (8.8) of [6]) and the third term is 

equal to ^"j^'i!^^^ (this follows from Theorem 4.1 of [9] by letting the /3,7 ^ 

and noting that if Il{dz) is the Levy measure of the process /, then (for all 
z > 0) n(— oo, —z] = v[z, oo). 

Our assertion is satisfied by replacing each of the three terms in (4.21) by 
the corresponding value indicated in the last paragraph and after some algebraic 
manipulations, which wc omit. 

(ii) The proof is immediate from (4.14) and (4.19). ■ 

* 

Now we turn our attention to computing Ca{x,M,T), Ex[ex.p{T~ r)], and 
* 

E^[T-r]. For each t>0 

Xt = {It,t<T'r}. (4.22) 

We note that the sample paths of a spectrally positive Levy process and a 

spectrally positive Levy process reflected at its infimum behave the same way 

* 

starting at any x > t until they reach level r, thus X behaves the same way in 
both cases. Let U be the apotential of the process X. For each x G (r, V] 

C^{x,M,T) = u"g*{x) (4.23) 

(M) 

Let I = 7 — M , as defined in Section 2.1. We note that this process is 
a spectrally positive Levy process with the Levy exponent 0m (^) = '/'(^) + 
6M, 6 >0. We denote its a— scale and adjoint a— scale functions by and 

, respectively. Note that wj;^^ is obtained from W^"^ , upon replacing the 
term a in (3.1) by a — M. 

* a 

Theorem. For a > 0, U is absolutely continuous with respect to the Leb- 
segue measure on (t, V], and a version of its density is given by 



u {X, y)= ^'^' '> - W'^^ {y-x). x,ye{T,V] (4.24) 



(M) 

Proof. For each t > 0, we define M t = I t — V. For any b G R, we 
define 7^ = ia£{t > : M t - M t > b} = in£{t > : Mt - Mt > b} and 
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7^ = inf{i > : M t - M t < 6}. For and Borel set A C (r, V] and x e (r, F] 
we have 



(M) (M) 

= P,{ 7 t-sup(( 7 «-y)vo) e <r-} 

= P,-v{Mt -M teA-V,t< T-_y} 

= Pv-^{Mt-Mt & V - A,t<T-_y] 

= Pv-^{Mt-Mt & V-A,t<T+_^} 
Using Theorem 8.111 (ii) of [6], the result follows. ■ 

The following theorem gives Laplace transform of the distribution of the 

* * 

stopping time T^^ and Ex\T^ t\,x G (r, V]. 

Theorem, (i) Let x G [r, y)and a G R+. Then we have 



^.[e-"^"1- !|^)|!^ (4.25) 



zti'{V-r) 
(ii) For X G [r, 1/) 



i;,[T-,]=W^M(V-T)-W^M(V-ar), (4.26) 

a: 

where, Wm{x) = y" WM{y)dy. 



Proof. Wc only prove (i), the proof of (ii) follows easily from (i) and is 
omitted. We have 

£^.[e""^"1 = l-au"\ry^{x) 

= 1 



= 1 



— a u {x, dy) 

Jt Z\'(V-t) 



_ Z^^\v-x) , Z-iV-r)-l ^ _ Z^^(V-x)-l 



M 

Z'^{V-x) + Z"{V-x) 



Z^^\V-r) 
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where the third equation follows from (4.24), the fourth equation follows from 
the definition of the function Z^^^ a 
equation after obvious manipulations. 



the definition of the function Z^^^ and the fifth equation follows the fourth 



Remark. When V = oo, for a > wc let rjjy^{a) = sup{0 : (j){9) — 9M = 

.(a) 
M 



ajsince Z^'"(a;) = 0(e''M(")^) as a; — J- oo, then we have 



r * n (x-t) 



^m(Q!)'(0) 

(x — t) 

= oo , if M< £;(7i). 

This is consistent with the well known fact about the busy period of the M/G/1 
queuing system. 



* * 
To compute Ex[cxp{—aTo)], we first observe that, for X < x < V, Tq = 

T~r almost everywhere. Hence Ex[exp{—aTo)] is given in (4.25). We now turn 
our attention to the case where x < A. Wc first consider the case where the 
input process is a spectrally positive L'evy process. 

Theorem. Assume that the input process is a spectrally positive L'evy 
process. For z > A, we define 



ha{x, dz) = / dyu"{x, y)v{dz - y), 
Jo 



u"{x,y) is defined in (4.6). 
Then, for a > 0, a; < A 



* 1 rv poo 

Eje-"^°] = ^— -[ / ^«(A - z)K{x, dz) + / K{x, dz)] (4.27) 

^-t) Jx Jv 



Z"(A 
Proof: We write 
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ha{x., dz)\ 



where the third equation follows since, given and , Tq — is equal 
to T~ almost everywhere. The fourth equations from (4.11), the last equation 
follows from the fact that Z"(0) = 1, and the definition of ha{x, dz). ■ 

The following theorem gives a result analogous to (4.27) when the input 
process is a spectrally positive L 'evy process reflected at its inflmum. 

Theorem. Assume that the input process is a spectrally positive L'evy 

process reflected at its inflmum. For z > \, let la{dz), La{z), and V^(A) be as 
deflned in (4.16), (4.17) and (4.18), respectively. Define 



Proof. The proof follows in a manner similar to the proof of (4.27), using 
(4.19), (4.20) and (4.25). ■ 

5. Special Cases 



In this section we consider the cases where the input process is a spectrally 
positive Levy of bounded variation, Brownian motion reflected at its inflmum 
and Wiener process. For the flrst case, we extend the results of Bae et al [5] who 
assumed that the input process is a compound Poisson process with a negative 
drift. We also simplify some of their results. For the second case, we obtain 
results similar to those of Attia [3] and Lam and Lou [7]. In the third case we 
obtain results similar to those of Zuckerman [10]. 




Then, for a > 0, a; < A 




(4.28) 
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[\j Assume that the input is a spectrally positive Levy process of bounded 
variation with Levy exponent described in (3.3), reflected at its infimum. Let 
/z = xv{dx) and assume that n < oo. For every x € R+, we define 
the probability density function f{x) = hHEiHH)!^ and J^{1 - e-^^)w(rfa;) = 
9iJ, Jq°° e~^^ f{x)dx. Define p = ^ and assume that p < 1, then, = 



oo 



where F{x) is the distribution function corresponding to / . Therefore, the 
0— scale function is given is given as follows 



^ oo 

W{x) = -^/9"F(")(x). 

n=0 

For a > 0, W^"^ is computed using (3.5) and the above equation. 



Remark, (a) Since, for all n > 0, F(")(a;) < [i^(x)]", then for all x, W{x) < 

r,if/9<l. 



i;~fj.F{x) ' 

(b) Bae et al [5] , treat the special case where the input process is a compound 
Poisson process with a negative drift. In this case, v{dx) = XG{dx), where 
A > and G is a distribution function of a positive random variable [0, oo), 
describing the size of each jump of the compound Poisson process. In this 

case, f{x) = and p = where G = 1 — G and m = G{x)dx, 

which is assumed to be finite. We note that their entities w{x) and E[L'^{X,t)] 
given in p. 521 are nothing but our C{x, A, 0) and E^Itx ], respectively. Using 
(4.12) and (4.15) we get the same result given in page 523 of this reference. 
Furthermore, their functions EIP'^ {^iT)] and E[L^ i^^''')] given in page 525 

are our C{x,M,t) and Ex[T^j-], respectively. Using (4.24) and (4.26) we get a 
simpler form for these entities. Furthermore, letting a = 0, in (4.19) we provide 
a simpler formula for the distribution of L{t) (the overshoot) given on page 525 
of their paper. 

(c) Assume that the input process is a gamma process with negative drift. 
The Levy measure is given by v{dx) — a^^-^dx^ a,b > 0. In this case, E{Ii) = 
C — |, which is assumed to be nonnegative and p = ^ < 1. It follows that 

f{x) = bj^ the right hand side is denoted by Ei{x) in p. 227 of [2], 

Direct integrations yields, F{x) = (1 — e~''^) +xf{x). 

(d) Assume that the input process is an inverse Gaussian process with a 
negative drift, and with Levy measure is given by v{dx) = -—^=e~^'^ 1'^'^ , 
(T,c > 0. It follows that E{1\) = — -j, which is assumed to be grater than 
zero. In this case p = — < \. In this case, /(x) = c /°° v{dy), and F{x) = 
erf(cy^72^)+x/(x). 
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^ii^ Assume that the input process is a Brownian motion with drift term 

fi £ R, variance term cr^, reflected at its infimum. We will show that the 
results of [3] and [7] follow from our results. In this case, the Levy measure 
V = Q, and from (3.1) we have, that for 6i > 0, ip{e) = -jiO + It fol- 

lows that, for a > 0, ri{a) = V^^E^tLhL^ Let 5 = \/2acr'^ + /x^, we have, 
^^^{x) = fe'*^/'^' sinh(2^) and = e^^/'^' (cosh(f|) - Hsinh(f|)) . We 

note that W^ix) is differentiable, and W^' {x) = + J^e"^/'"" cosh(ff ), 

it follows that ^('^"'/^^^^ = Jth((X|) ^ • Substituting the values of Z(")(A - 
a;), W(«)(A - x) and ^^M: in (4.14), we have, for a > 0, a; < A 



cosh 



(A - x)5 



■ sinh 



(A - x)5 



^ /x + (5coth(^)^ 

Kw\x) = ^-^ 



Case 1. /Lt#0: It follows that, for a; > 0, W{x) = 

and W{x) = ^(e^"^/"^' - 1) - ^- Substituting the values of W{\ - x), W{\ ■ 
x),W{\) and W' [X) in (4.15) we have, for a; < A, 



X — X a 



2 r 



(2) 



We note that, W^(")(0) = 0, in (4.12) is absolutely continuous with 
respect to the Lebesgue measure on [0,A) and for y e [0,A), W'^°'\dy) = 
W°''{y)dy. Substituting the values of W^^^X - x), W"' {X), W^"\y - x), and 

(2) (2) 

W" (y) in (4.12) wc get a version of the the density of U", denoted by u". Thus 
, Cq(x, A,0) is computed using (4.3). 



Let ti = l^-M,S=y 2acr^ + *n , we note that W^{x) = |e'^^/'^' sinh(f|). 
We note that the input process is continuous, Y _^ = X < V, almost every 

where. Therefore, the term Ca{{ZA A V),t, M) in (2.6) reduces to Ca{X, r, M) 

To 

which is computed using (4.23). Furthermore, 



where Ex[e ""^a] is given above and Ex[e "-^^ ] is given in (4.25). 
Let A* = F - A and t* = V -t, then 

Er[T,] = EAT,]+Ex[T;], 



Note that 
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A — T 

■ + 

2/t 



E^[T-] = — + — 

fx* *^ 



where the last equation follows from (4.26) after some tedious calculations which 
we omit. 



Case 2. = 0: In this case, 5 = V^aa^, letting — )• 0, in the corresponding 
equations in case 1 above, we have 



cosh((A — x)5/a ) — a 



, sinh ((A — x)5/a"' 



- . _ 2^ 

3\ _2' 



coth(A|) 



t2 



and Ex[Tf] is obtained by replacing /x* by — M in the last equation of case 1. 

(^111^ Assume that the input process is a Brownian motion with drift term 
fj, > and variance parameter tr^. Substituting the values of W^"^x), Z^"^x), 
given in (ii) in (4.7) we have, for x < A, Ex[e~°''^^] = exp {{S — fx){x — X)). 
Substituting r?(0) = ^, ^{e"^^^/"^ - 1) and ^(e^''^/'^' - 1) - ^ '^i^) and 

W{x), respectively, in (4.8) we have, for x < A, E^[T+] = These results 

are consistent with the results of Zuckerman [10], p. 423. The computations of 
the other entities in the cost functionals (2.7) and (2.8) can be obtained in a 
manner similar to those discusses in (ii) with obvious modifications. 

References 

[1] Abdel-Hameed, M. (2000). Optimal control of a dam using PjJ^ policies 
and penalty cost when the input process is a compound Poisson process with 
positive drift. J.Appl.Prob. 37, 508-416. 

[2] Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical 
Functions. Dover, New York. 

[3] Attia, F. (1987). The control of a finite dam with penalty cost function; 
Wiener process input. Stochastic Processes and Their Applications 25, 

289-299. 

[4] Bae, J, Kim, S. and Lee, E.Y. (2002). A Pj^ policy for an M/G/1 
queueing system. Appl. Math. Modelling 26, 929-939. 

[5] Bae, J, Kim, S. and Lee, E.Y. (2003). Average cost under - policy 
in a finite dam with compound Poisson input. J.Appl.Prob. 40, 519-526. 

[6] Kyprianou, A. E. (2006). Introductory Lecture Notes on Fluctuations of 
L 'evy Processes with Applications. Springer Verlag. 

[7] Lam, Y. and Lou, J.H. (1987). Optimal control of a finite dam: Wiener 
process input. J.Appl.Prob. 35, 482-488. 



17 



[8] Lea, E.Y. and Ahn, S.K. (1998). policy for a dam with input formed 
by a compound Poisson process. J.Appl.Prob. 24, 186-199. 

[9] Zhou, X. W. (2004). Some fluctuation identities for L'evy processes with 
jumps of the same sign. J.Appl.Prob. 41, 1191-1198. 

[10] Zuckerman, D. (1977). Two-stage output procedure for a finite dam. 
J.Appl.Prob. 14, 421-425. 



18 



